A Wrapper Method for Cost-Sensitive Learning via Stratification

Many machine learning applications require classifiers that minimize an asymmetric loss function rather than the raw misclassification rate. We introduce a wrapper method for data stratification to incorporate arbitrary cost matrices into learning algorithms. One way to implement stratification for C4.5 decision tree learners is to manipulate the weights assigned to the examples from different classes. For 2-class problems, this works for any cost matrix with zero values on the diagonal, but in general, for k > 2 classes, it is not sufficient. Nonetheless, we ask what is the set of class weights that best approximates an arbitrary k k cost matrix. We test and compare the new wrapper method against several heuristic methods. The results show that the best method is the wrapper method that directly optimizes the loss using a hold-out data set.